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Abstract 


Additional  quantitative  results  are  presented  for 
the  existence  of  optimal  decision  rules  and  stochastic 
stability  for  linear  systems  with  white  random  parameters 
with  respect  to  quadratic  performance  criteria,  by  exam- 
ining a specific  version  of  a multivariable  optimization 
problem. 
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This  paper  considers  the  optimal  stochastic  control  of  a multi- 
variable  linear  system,  with  a specific  structure,  with  respect  to  a 
quadratic  index  of  performance.  The  system  dynamics  are  described  by 
a linear  vector  difference  equation  in  which  both  the  system  matrix  (A) 
and  control  matrix  (B)  are  multiplied  by  white,  possible  correlated, 
scalar  random  sequences. 

A threshold  condition  involving  the  maximum  eigenvalue  of  the 
j . system  matrix  A and  the  means,  variances,  and  cross  correlations  of  the 

white  parameters  is  obtained.  If  the  threshold  condition  is  violated, 
then  there  does  not  exist  an  optimal  solution  to  the  infinite  horizon 
optimization  problem,  and  the  resultant  closed-loop  system  is  not  stable 
in  a mean  square  sense. 

The  results  of  this  paper  represent  another  manifestation  of  the 
Uncertainty  Threshold  Principle  (UTP)  reported  by  Athans,  Ku,  and 
Gershwin  in  [1]  and  represents  a specific  multivariable  extension  of  the 
scalar  results  reported  in  [1] . It  also  generalizes  the  results  of 
Kutayama  [2],  which  dealt  with  control -dependent  white  noise,  to  the  case 
of  simultaneously,  possibly  correlated,  state-  and  control -dependent  white 
noise  parameters. 

2 . PROBLEM  FORMULATION 


Consider  a linear  stochastic  discrete-time  system  whose  dynamics  are 
described  by  the  following  vector  difference  equation 


(1) 


x(t+l)  = Y(t)A  x(t)  + 6(t)B  u(t)  + £(t) 

where  x(t)  is  the  n-dimensional  state  vector,  jj(t)  is  the  m-dimensional 
control  vector,  and  C.(t)  is  white  gaussian  process  noise.  Assume  that 
A and  B are  constant  matrices  of  appropriate  dimensions  and  that  [A,  B] 
is  a controllable  pair. 

Further  assume  that  the  scalars  V (t)  and  6(t)  are  gaussian  white 
random  sequences  (uncorrelated  in  time)  with  known  stationary  statistics. 


More  precisely,  we  assume 

E{y(t)}  = Y ; e{ (Y(t)  - Y)(Y(T)  - Y)>=  r 6 (t,x)  (2) 

E{6(t)}  = X ; E{(6(t)-7)  (6 (T)  -J)}=ii(t,T)  (3) 

E{(Y(t)  - Y)  ( <5 ( t)  - 7)}  = A 6(t,t)  (4) 

Et£(t)}  = 0 ; EU(t)i' (T)}  = S 6 (t,T)  (5) 


where  6(t,t)  is  the  Kroenecker  delta  (6(t,r)  * 1 if  t * T,  6(t,T)  = 0 if 
t ^ T) . Furthermofre  assume  that  the  process  noise  £(t)  is  mutually  inde- 
pendent of  the  random  parameters  Y(t)  and  6(t). 

We  consider  a standard  quadratic  cost  functional 
T 

J = e{^  ^ 1 x'  (t)£  x(t)  + u'  (t) R u(t)  } (6) 

t=0 

1/2 

where  2.  is  positive  semidefinite,  R is  positive  definite,  and  , A]  is 

an  observable  pair. 

Under  the  assumptions  that  we  can  measure  the  entire  state  vector 
x(t)  exactly,  at  each  instate  of  time,  we  wish  to  find  the  causal  optimal 
control  sequence  u(0),  u(l) , u(t) , ...  which  minimizes  the  quadratic 
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cost  (6) . 

Remark  1 The  problem  considered  by  Katayama  [2]  corresponds  to  the 
case 

7 = 1,  r = 0,  A = 0,  H = 0 (7) 

3.  SOLUTION 

The  problem  can  be  readily  solved  by  stochastic  dynamic  programming 
[3]  for  any  fixed  value  of  the  planning  horizon  time  T.  The  derivation 
is  straightforward  and  hence  omitted.  Only  the  results  are  stated. 

The  optimal  control  is  obtained  using  linear  state  variable  feedback, 

i.e. 

u*(t)  = -G(t)x(t)  (8) 

The  optimal  m*n  feedback  control  gain  matrix  G(t)  is  given  by  the  formula 

G(t)  = [R  + (62  + A)B’K(t+l)B]'1(Y  6 +A)B'K(t+l)A  (9) 

The  nxn  matrix  K(t)  satisfies  a recursive  matrix  equation  of  the  form 

K(t)  = (y2  +DA'K(t+l)A  + ^ 

- (Y  6 + A) 2A'K(t+l)B[R  + (62  + A)B,K(t+l)B]~1B’K(t+l)A 

(10) 

with  K (T)  = 0. 

Remark  2 The  recursion  (10)  will  be  referred  to  as  the  UTP  matrix 
equation;  it  is  similar  to  a matrix  Riccati  equation.  However,  unlike 
Riccati  equations  it  cannot  be  related  to  a coupled  set  of  linear  equa- 
tions. Under  our  assumption  the  positive  definite  matrix  K(t)  exists 
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and  is  bounded  for  all  finite  planning  horizon  times  T. 

The  optimal  cost  (6)  is  given  by 

T 

J* (x  (0)  , T)  = £ X*  (0)K(0)x(0)  + i X^K(t)5  (11) 

t=0 

4.  THE  INFINITE  HORIZON  CASE  (T  -*■  °°) 

The  interesting  results  occur  as  one  analyzes  the  infinite  horizon 
case,  T •*  °°,  so  as  to  examine  the  existence  of  an  optimal  solution  and 
the  stabilizability  of  the  stochastic  system  [1],  [2],  Once  more  we 
shall  show  that  there  exists  a threshold  condition  which  provides  a 
dividing  line  between  existence  and  non-existence  of  optimal  solutions 
to  the  problem  as  T + ®.  We  summarize  the  main  result  as  follows. 


Theorem  1 (Uncertainty  Threshold  Principle) 

An  opcimal  solution  exists  for  the  problem  stated  in  Section  2 as 
T + ®>  if  and  only  if 

max  |X.  (A) | < 1/B  > i = 1,2, ... ,n  (12) 

i 1 


where  3 is  defined  by 


e2  = Y2  + r - -6'  -*  A--  ■ > 0 


72 


+ A 


(13) 


and  max  |X^(A)|  denotes  the  magnitude  of  the  maximum  eigenvalue  of  the 
i 

constant  system  matrix  A in  the  system  dynamics  (1) . 

Before  we  present  the  proof  of  the  theorem  it  is  important  to  make 


some  remarks. 
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Remark  3 In  the  case  of  non-random  parameters  (T  = A = A = 0) , 3=0. 

This  means  that  given  our  assumptions  of  [A,  B]  controllability  and 
1/2 

[A,  y ] observability,  one  can  always  solve  the  infinite  horizon  opti- 
mal control  problem  independent  of  the  (open  loop)  eigenvalues  of  A. 

On  the  other  hand,  as  the  variances  r and  A of  the  random  parameters 
increase,  then  8 increases,  and  the  value  of  1/8  defines  the  radius  of 
a shrinking  disc  which  must  contain  all  the  open-loop  eigenvalues  of  A 
in  order  for  the  problem  to  have  a solution. 

Remark  4 If  the  condition  (12)  is  violated,  i.e.  if 

max  |Xi(A) | > 1/8  (14) 

i 

then  there  is  no  solution  to  the  optimal  control  problem,  and  one  cannot 
stabilize  (in  a mean  square  sense)  the  system  (1) . Under  these  condi- 
tions the  optimal  cost,  J*,  as  defined  by  (Hi  undergoes  exponential 
growth  as  T increases 

max  | A ( 8 A ) | T 

J*(T)  ^ c e 1 ; c = constant  (15) 

Because  of  the  explosive  growth  of  the  optimal  cost,  then  only  short 
term  (small  T)  decisions  make  sense;  see  also  [1] . 

As  in  the  scalar  case  [1],  even  if  condition  (14)  holds,  the  con- 
trol gain  matrix  G(t) , see  eq.  (9) , remains  well  behaved  and  bounded 

G ■ lim  fry  * [B'K(t+l)B] ~1B,K(t4-l)A  (16) 

||K(t+l)||-*«  S'  +A 


-7- 


Next  we  present  the  details  of  proving  Theorem  1.  We  remark  that  the 
proof  essentially  uses  algebraic  manipulations  and  known  properties  of 

discrete  Lyapunov  and  Riccati  matrix  equations. 

> 

The  main  idea  of  the  proof  is  to  examine  the  behavior  of  lim  K(t) 

ip-»O0  1 

or  the  behavior  "backward  in  time"  of  the  UTP  matrix  equation.  The 
arguments  are  similar,  but  not  identical,  to  those  given  in  [2], 

For  the  sake  of  notational  convenience  define  the  scalars 

= Y2  + r,a2  = (y  J + A)2;  a3  = l/62  + A (17) 


Then  the  UTP  equation  (10)  can  be  written  as 


K(t)  = ^A'KIt+llA  + £ 

- a2A’K(t+l)B[R  + — B,K(t+l)B)~J'B,K(t+l)A 
a3 


(18) 


From  Eqs . (13)  and  (17)  one  sees  that 


& “ “l  ’ a2a3 


(19) 


By  adding  and  subtracting 


otjC^A*  K(t+1)A  (20) 

to  the  right  hand  side  of  Eq.  (18) , and  some  algebra,  Eq.  (10)  reduces 
to 

K(t)  = S2A’K(t+l)A  + 2 

+ a2a3A' (K(t+1)  - K(t+l)B(a3R  + B’K(t+l)B]_1B'K(t+l) ]A 

(21) 

Attention  is  focused  to  the  matrix 

M(t+1)  * K(ttl)  - K(t+l)B[d3R  + B’K(t+l)B)_1B’K(t+l) 


(22) 
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Such  matrices  arise  naturally  in  the  Riccati  equation  of  standard  linear 

quadratic  problems  where  the  control  weighting  matrix  is  a^R.  Under  the 

1/2 

given  assumptions  of  [A,  B]  controllability  and  [A,  £ ] observability 

it  is  known  that  that  [4], [5]* 

(a)  M(t+1)  = M’  (t+1)  > 0 (23) 

(b)  There  exists  a bound 

L > M(t)  all  t (24) 

Since  M( t+1)  is  positive  definite,  so  is  a^^A'Mft+l) A.  Hence  we  obtain 

K(t)  > B2A’K(t+l)A  + g.  (25) 

From  (25)  it  is  obvious  that  if  any  eigenvalue  of  (6  A)  is  greater  than 
unity  then  K(t)  grows  without  bound  backward  in  time,  that  lim  K(t) 

rp-*oo 

does  not  exist  and  that  the  optimal  cost  undergoes  exponential  growth 
as  indicated  by  (15).  On  the  other  hand  from  (24)  and  (25)  one  obtains 
that 

K(t)  < B2A'K(t+l)A  + 2 + C^C^A'L  A (26) 

Hence  if  all  eigenvalues  of  (0  A)  are  less  than  unity,  the  right  hand  side 
of  the  recursion  (26)  will  approach  a bounded  constant  solution  matrix 
and  so  will  K(t) . Hence,  the  limiting  solution  lim  K(t)  is  well  defined. 

We  remark  that  the  above  proof  requires  that  B is  nxn  and  nonsingu- 
lar, as  required  in  the  Corollary  of  [2] . However  we  believe  that  this 
is  a sufficient,  but  by  no  means  necessary,  condition?  it  could  probably 

* The  notation  A > B (A  >_  B)  means  that  A-B  is  positive  definite  (A  - B 
is  positive  semidef inite) . 
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k 


be  removed  by  a more  detailed  analysis  of  the  UTP  difference  equation. 

5.  CONCLUSIONS 

The  quantitative  results  of  the  Uncertainty  Threshold  Principle 
have  been  extended  to  a special  case  of  a multivariable  control  problem, 
generalizing  the  results  in  [1]  and  [2]. 
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